The invention applies more particularly to the situation where an input block has some pixels with all colour components (non-sub-sampled pixels) and some pixels with less colour components (sub-sampled input pixels) because of a sub-sampling scheme sub-sampling Chroma values (e.g. a 4:2:0 YUV block), and where such input block needs to be up-sampled into a pre-encode block of pre-encode pixels having each the same number of pixel or colour component values (i.e. no subsampling, e.g. a 4:4:4 YUV block), before it is processed.
The sub-sampled, and thus missing, pixel components are referred below to secondary pixel components, while the main pixel components are those present in both input and pre-encode pixels. In the example above, the main pixel component is the Luma colour component and the secondary (and missing in the sub-sampled input pixels) pixel components are the Chroma colour components.
An example of such situation requiring up-sampling before processing is the so-called palette coding mode, to which it is now made reference for illustrative purposes.
As known, the palette coding mode uses a palette comprising a set of entries associating respective entry indexes with corresponding sets of pixel colour component values. The palette makes it possible to build a block of indexes to be used to represent the pre-encode block of pre-encode pixels, i.e. to encode the input block of pixels.
A palette in this document is defined as a look up table having entries, or “elements”, associating an index with a value of a pixel. Typically, the value of a pixel is constituted by the value of each colour component associated with the pixel, resulting in a colour palette. Well known colour spaces are RGB representing Red, Green and Blue components and YUV (or YCbCr) representing one Luma and two Chroma components.
When encoding an image in a video sequence, the image is first divided into coding entities (also known as “coding structures”) of pixels of equal size referred to as Coding Tree Blocks (CTBs). The CTBs may be grouped into other coding structures having a higher hierarchical level, such as slices and/or tiles. In other words, the image is recursively divided into hierarchical coding structures or coding entities.
The size of a Coding Tree Block is typically 64 by 64 pixels. Each Coding Tree Block may then be broken down into a hierarchical tree of smaller blocks whose size may vary and which are the actual blocks of pixels to encode. These smaller blocks to encode are referred to as Coding Units (CUs).
The encoding of a particular Coding Unit involves competition between predictive coding modes, including the well-known INTRA coding mode, the well-known INTER coding mode, and the Palette coding mode.
With the Palette coding mode, it is possible to define a representative block for a given Coding Unit as a block of indexes (also known as “levels”) from a palette: for each pixel location in the Coding Unit, the said representative block contains the index associated with a pixel value in the Palette which is the closest to the value of the pixel having the same location (i.e. colocated) in the coding unit. However, this palette-based algorithm of selecting the closest palette entry is only used at the encoder in HEVC SCC: there is no need to know said algorithm in order to parse or decode a bitstream at the decoder.
Typically, “closest” means with the lowest distance using a particular metric distance such as the sum of absolute, or the square of, differences of component values, in case of lossy encoding. In the particular case of lossless encoding, this means the palette entry should be selected as equal to the pixel by the encoder. In the following, “correspond to” or “match” is used to mean either “is equal” when in lossless coding, or “is the closest” otherwise.
In the recent version of HEVC SCC, no residual between the original pixel block and the corresponding palette-based representative pixel block is provided. To avoid high quality decreasing in the encoded image, an “escape-coded” feature has been introduced to encode the pixels, the values of which do not match a pixel value of an entry of the Palette. It means, in lossless encoding, that no palette entry is equal to the pixel value. Whether a CU is coded using escape coding is signalled by a flag. In addition, a specific index in the Palette is used to signal an “escape-coded” pixel; and the quantized value itself of the escape-coded pixel is directly encoded in the bitstream, the quantization depending on a quantizer step transmitted at the CU-level, only if the CU is signalled as using escape coding. In case of lossless encoding, the quantizer step is 0, meaning no quantization. The quantization is what is defined in the HEVC standard as the transform-bypass quantization, and the quantized values are encoded using truncated binary codes.
As currently proposed, the palette coding mode may be used for non-4:4:4 content, i.e. for input blocks conforming to an input sampling scheme having subsampling of at least one of the colour components forming the pixels, i.e. in which at least one sub-sampled input pixel includes the main pixel component or components but has secondary pixel component or components missing. This is for instance the case in 4:2:2 or 4:2:0 YUV signals where the Chroma components are subsampled and thus missing for some of the input pixels.
However, for this type of non-4:4:4 contents, the decoder first decodes the data as 4:4:4, and then discards the unused Chroma components to match the input format. It means that the encoder has to perform the reverse approach, by up-sampling the input block of pixels (e.g. in 4:2:2 or 4:2:0 sampling scheme of corresponding 4:2:2 or 4:2:0 Chroma formats) into a pre-encode block of pre-encode pixels conforming to a pre-encode sampling scheme having no subsampling of the colour components forming the pixels, e.g. 4:4:4 sampling scheme, i.e. in which all pre-encode pixels have all main and secondary pixel components.
The required up-sampling step introduces suboptimal computation when applying the palette coding mode to encode the pre-encode block of pixels. In particular, lossless encoding introduces stringent requirements that can actually be alleviated when the content is not 4:4:4.